make trtllm attn backend's init_forward_metadat non blocking by cicirori · Pull Request #13802 · sgl-project/sglang

cicirori · 2025-11-23T15:01:28Z

Motivation

This is a minor performance fix. The original implementation caused a CPU-to-GPU copy, which in turn introduced a host–device synchronization point.

Format your code according to the Format code with pre-commit.
Add unit tests according to the Run and add unit tests.
Update documentation according to Write documentations.
Provide accuracy and speed benchmark results according to Test the accuracy and Benchmark the speed.
Follow the SGLang code style guidance.
Work with maintainers to merge your PR. See the PR Merge Process